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(57) Abstract: A method provides bookmarks for indicating elements or portions of an information content that are likely to be of 
great interest to an audience. A broadcast station can these boolunarks for sale of lease to a third party for inserting data into the 
information content at the bookmarked locations. The third party can insert advertisements in the information content close to the 
indicated portions that the audience is likely to appreciate. 
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Content with bookmarks obtained from an audience's appreciation 



The invention relates to a method of indicating particular portions in electronic 
information content. The invention is relevant to content broadcast stations which can offer 
these bookmarks for sale of lease to a third party for inserting data into the information 
content at the bookmarked locations. 
5 The invention also relates to an apparatus for determining and retrieving the portions 

in the information content. 



US patent 6,011,895, incorporated herein by reference, discloses a method of 
10 retrieving programs, segments from a program, and/or segments from a plurality of 

programs. The method enables a user to establish content preferences by means of at least 
one keyword. Each segment is analyzed as to subject miatter and is assigned a keyword. The 
keywords provide an effective means for retrieval of a desired segment and enable the user to 
control what is being played out Keyword indexing of the segments provides the capability 
15 of inhibiting the viewing of undesirable subject matter, or of assisting in the retrieval of 
desirable subject matter. The same patent document describes a system for generating a 
transparently seamless video program matching the viewers* pre-established content 
requirements. 

The known method classifies segments of programs or entire programs according to 
20 their contents. The known method does not address the expected impact of a segment of a 
program on an audience. The inserted keywords describe the content of the segment in a 
neutral manner, i.e., without considering the relative degree of interest to a t3^ical audience 
or without annotating highlights in the program. The inserted keywords do not give any 
indication whether or not the audience will like the segment when the segment is perceived 
25 within the program. The keywords are used to deliver selected programs whose content 

satisfies the usct's preferences. Such a method relates to the selection in advance of programs 
to be shown to the user. 
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It is an object of the invention to efficiently select portions of given information 
content that are likely to be of the most interest to an audience. Another object of the 
invention is to mark these portions in the information content that are likely to raise the 
highest interest or will attract the highest attention. Another object is to create income from 
5 maridng these portions. 

To this end, the invention provides a method of enabling to use a bookmark that 
indicates an element in information content, the element being selected on the basis of an 
appreciation, expected or tested, from an audience perceiving the information content. 

The expression "information content" as used herein indicates what a user perceives 

10 when the information content is developed, played out over time.. An aspect of the invention 
aims at determining and referencing elements or portions of the information content that the 
audience is likely to appreciate. An element can be a portion of the information content 
perceived by the audience during a certain length of time. The portion can have a length of a 
few seconds, a few minutes or can be the information content occurring at a particular 

15 instant, e.g., a snap shot or a frame. For example, in a movie an element can be a scene of the 
movie, a sound or an object that appears in an image. The word ^'appreciation'", as used 
herein, nnueans a favorable critic estimate. The term "bookmark" as used herein refers to an 
identifier indicating a specific element in the information content. Thus, a bookmark is a 
means to label in the information content an element that the audience likes or is expected to 

20 appreciate. An advantage of the invention is to permit to determine in advance portions of the 
information content that are likely to be judged positively by the audience. The audience can 
be composed of members tiiat aie arbitrarily chosen. Alternatively, the targeted audience is a 
group of specific spectators chosen on the basis of, e.g., demographic or geographic criteria. 
Compared to the known method in the background art section, a program is delivered in its 

25 entirety to the audience. The invention does not consider whether a specific program is in 
accordance with the audience requirements and does not aim at selecting parts of the 
segments that should or should not be delivered to the audience. An object of the invention is 
to adopt a positive approach towards a program or other information content as a whole. The 
method of the invention comprises selecting and marking only the portions of the information 

30 content that are likely to be preferred* 

The invention also provides a method of offering for sale or lease a right of inserting 
data into the information content on the basis of an appreciation of an audience perceiving 
the content..Such right may be sold to a service provider. Thus, the service provider can 
insert data such as advertisement into the information content. The service provider inserts 
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advertisements into the element or in the vicinity of the element of the information content 
that the audience is expected to appreciate. Thus, the advertisements have a greater chance to 
be perceived by the audience since they are inserted in the information content where the 
attention level of the audience is likely to be high. 
5 ' The invention is explained in further detail, by way of example, and with reference to 

the accompanying drawing wherein: 



Fig. 1 is a diagram explaining the appreciation from the audience perceiving the 
10 information content; 

Fig. 2 shows an audience perceiving content and an apparatus of the invention; arid 
Fig. 3 is a diagram explaining the appreciation from two different audiences 
perceiving the same information content. 

15 " 

Fig. 1 is a diagram relating to an audience, which perceives information content 
originating from a signal SI. In the following the expression "content" refers to "information 
content**. The content is, e.g., a video program or an audio program that can be watched or 
heard on a television, a radio or via the Internet on a computer, for example. Hie signal SI is 

20 the linear physical representation of the content CI in the time domain. The signal SI is 
typically a physical quantity varying with time. 

A bookmark Bi indicates an element of the content. Tliis element is determined from 
the appreciation from the audience perceiving the content The term ^'appreciation** as used 
herein includes the concept of a favorable critic estimate given by the audience. The 

25 bookmark Bi is either inserted in the signal SI or is provided separately from the signal SI as 
will be explained further below. When the bookmark Bi is inserted into the signal SI, the 
bookmark Bi allows to identify an element in the content SI and can be identified and 
recognized as a bookmark within the signal SI. 

Hie audience can be arbitrarily chosen and can comprise several auditors or a sin^e 

30 auditor. Alternatively, the information content may be intended for a specific targeted 

audience. Therefore, in a first place, a test audience is selected, representative of the specific 
audience to which the content is actually targeted. The content is played to this test audience 
to determine the test audience's appreciation and to determine the elements appreciated by 
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the test audience that are likely to be appreciated by the specific audience. Thereafter, the 
content can be played to the specific audience to which the content CI is targeted. 

A diagram 100 in Fig. 1 gives the evolution of a measure of the audience's 
appreciation along the evolution of the content over time. The diagram 100 gives on the 
5 vertical axis a measure of the appreciation in an arbitrary unit The ^preciation can be 

measured by the attention level of the audience, which can be the number of people showing 
interest in the content, e.g., by applauding, laughing, etc... The appreciation can also be 
measured by the intensity of the noise made by the audience. This number or intensity varies 
along the content. The appreciation of an element of the content by the audience is measured 

10 in comparison with other elements of the same content Indeed, the varying appreciation 
indicates which portions are relatively interesting. 

The diagram 100 gives on the horizontal axis the reference X indicating locations of 
elements along the content. Any element in the content is identified by its location Xi in the 
content or correspondingly in the signal SI. Since the signal SI is at the origin of the content, 

15 there is a mutual correspondence between locations in the signal SI and locations in the 

content. An element in the content is for example referenced by the play time passed between 
the start of playing out the content and the occurrence of the element Then, in this example, 
the reference Xi of an element in the content is the moment in time at which the element is 
perceived by the audience. In another embodiment, the signal SI, and as a consequence the 

20 content, is segmented into successive segments and a single measure of the measured 

appreciation is attributed to an entire segment The measure of the appreciation attributed to a 
given segment is, for example, an average of the appreciation of the audience measured over * 
the entire segment For example, a video program is divided into scenes and the appreciation 
by the audience is graded from 1 to S. A scene that is graded S is among the most appealing 

25 scenes. A scene graded 1 is a scene that the audience relatively dislikes, i.e. in comparison 
with other scenes of the program. 

The diagnan 100 shows three peaks occurring at locations XI, X2 and X3 that 
indicate those elements in the content that the audience prefers. An element could be an 
image, a scene, an action in a video or video/audio content or can be a tone, a sequence of 

30 tones in an audio content a movement in a symphony or a part of a song's lyrics, etc. . 
Bookmarks Bl, B2 and B3 are inserted into the signal SI at the locations XI, X2, X3 
representative of the peaks of the appreciation. One may choose to insert the bookmarks Bl, 
B2, B3 at the exact locations XI, X2, X3 of the peaks or one may choose to insert the 
bookmarks Bl, B2, B3 an arbitrary length after or before the occurrence of the peaks of 
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interest. Alternatively, bookmarks Bi can also be inserted at locations in the signal SI for 
which the appreciation goes above a predefined threshold. 

Fig. 2 shows an apparatus 200 of the invention. An audience 210 perceives content C. 
The audio and video playing apparatus 200 allows rendering the content C from a signal S. 
5 The apparatus 200 of the invention can have different functionalities. 

A first possible functionality of the apparatus 200 is to allow playing out of the content C. To 
this end, the apparatus 200 comprises a play unit 201 that allows rendering the content C 
from the signal S to the audience 210. For example, the apparatus 200 comprises a computer, 
a radio or a home theater and the unit 201 comprises a VCR, a CD player, etc. . . The signal S 

10 is stored on a video tape, a hard-disk drive or any other storage medium. 

A second possible functionality of the apparatus 200 is to allow the recording of the 
signal S. The apparatus 200 comprises a recording unit 202 for recording the signal S on an 
appropriate storage medium. The apparatus 200 comprises, for example, a Video Cassette 
Recorder (VCR), a CD recorder, a video camera or any other data recorder. The signal S may 

15 be received from an external send^ such as a broadcaster or it may downloaded via a data 
network, e.g., the Internet. The apparatus 200 can also comprise a video/audio camera or an 
audio recorder for recording the signal S obtained from a live information content or a live 
event. Such a live information content is symbolized by the house 230. 

The apparatus 200 further comprises a marker 203 for inserting the bookmarks Bi in 

20 the signal S. There are different ntianners of inserting the bookmarks Bi in the signal S as 
explained hereinafter. 

A first manner is to insert a bookmark Bi in the signal S in response to a command of 
the audience 210 perceiving the content C. 

The audience 210 can be an end-user at home watching a program or content C on the 

25 television, the content C being obtained from a broadcaster. The user 210 records the signal S 
on a storage medium by means of the apparatus 200. The bookmarks Bi may be inserted 
while the signal S is actually being recorded. In another example, the signal S is not 
broadcast but obtained from the live event 203 of Fig. 2 and in this case the apparatus 200 
comprises a video camera. 

30 Alternatively, the signal S is prerecorded, played and the bookmarks Bi are inserted in 

the prerecorded signal S. The apparatus 200 allows to render the content C to the user 210 
from the recorded signal S. When perceiving the content C, the user 210 sends a command to 
the apparatus 200 whenever he/she is particularly interested in an element of the content C, 
The command is possibly sent when the user 210 presses a button on a remote control. In 
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response to this command, the marker 203 inserts the bookmark Bi into the signal S. The 
bookmark Bi is possibly inserted at the location corresponding to the element that is currently 
being played or is possibly inserted at a location determined by a predefined protocol. When, 
for example, the content C and the signal S are divided into segments, such a protocol can 
5 require that the bookmark Bi be inserted at the beginning of the segment whose content is 
currently played. 

A second manner is to insert automatically a bookmark Bi into the signal S on the 
basis of known statistics of the audience's appreciation. Indeed, it may be known in advance 
from previous experiences with other content semantically close to that of the content C that 

. 10 certain elements of the content C have a greater chance than others to be appreciated by the 
audience 210. The bookmarks Bi are inserted into the signal S at the locations Xi indicating 
these elements. When considering content C being a football match, for example, a bookmark 
Bi is automatically inserted into the signal S at the location Xi indicating a goal being scored 
or a replay of the goal. Indeed, in this example of a football match, goals are known, from 

15 past experiences, to be the elements for which the attention level of the audience 210 is likely 
to be high. 

A third manner to insert a bookmark Bi is based on the measured appreciation of a 
test audience. A content C may be destined to be broadcast to a targeted audience 210, whose 
demographic characteristics are known. A broadcaster may decide to insert the bookmarks Bi 

20 before broadcasting the content C to the targeted audience 210. To this end, the broadcaster 
arbitrarily selects a test audience that is representative of the targeted audience 210. The 
content C is rendered to this test audience and the bookmarks Bi are inserted into the signal S 
on the basis of the appreciation from the test audience. For example, a television show may 
be played live to a selected test audience and simultaneously the signal S is recorded. The test 

25 audience's appreciation is measured as the signal S is recorded so that the bookmarks Bi can 
be inserted into the signal S while recording. 

The bookmarks Bi can have several uses as shown hereinafter. 
Advantageously, the broadcaster offers for sale the locations Xi of the bookmarks Bi 
in the signal S to a service provider. Such locations Xi indicate the most valuable possible 

30 portions of the content C for inserting advertisements since these portions are likely to be 
seen by a large number of auditors. The bookmarks Bi in the signal S are used to control the 
insertion of advertisement data ADi into the signal S. The obtained signal comprising the 
embedded data ADi leads to a content that comprises advertisements corresponding to the 
data ADi. For example, when the content C is a video content, an advertisement may appear 
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as a visual element or logo inserted in the displayed image identifying the provider's service 
or goods. An advertisement may also be a short video clip inserted in die content C at the 
bookmark location- The content C is dien interrupted for the duration of the video clip. The 
advertisements are inserted in the content C before the content is broadcast 
5 The broadcaster adjusts the price of a given location Xi indicating an element in the 

content C on the basis of the relative value of the measured appreciation for this element. The 
higher the measured value of the appreciation is, the higher the price can be. 

The user 210 may also use the bookmarks Bi for scanning the content C. Once the 
bookmarks Bi are inserted in the recorded signal S, the bookmarks Bi are possibly used to 

10 retrieve elements in the content C. Advantageously, the signal S is digital and the play unit 
201 allows a functionality of jumping from one bookmark Bi to another when scanning the 
content C. This functionality permits the user 210 to play only the elements in the content C 
that he prefers. By pressing a key on a remote control the user can navigate through the 
digital signal S in either direction from one bookmark to another. The play unit 201 allows to 

15 go to from one bookmark to another that are not necessarily consecutive. 

In another embodiment not shown here, the bookmarks Bi may not necessarily be 
embedded in the signal S. A separate signal comprising the bookmarks Bi for indicating the 
locations Xi in the signal S can also be supplied. The separate signal may be processed so 
that the user retrieves from the signal S, into which no bookmarks are inserted, elements in 

20 the contmt C. In this embodiment, the apparatus 200 allows to combine the signal S and the 
sq)aiate signal comprising the bookmarks Bi that indicate the locations Xi. From this 
combination of the signal S and the separate signal, the apparatus 200 allows a user to access 
the elements in the content C indicated by the provided bookmarks Bi. The separate signal 
comprising the bookmarks Bi is for example provided through the Internet. 

25 Fig. 3 illustrates another embodiment of a method of the invention. The content C is 

submitted to a first audience 301 and a second audience 302, different from the audience 301. 
The auditors may vary from the audience 301 to the audience 301 according to their age, 
tastes or whether they are female or male. A first diagram 31 1 is obtained showing the 
evolution of a measure of the appreciation by the audience 301 along content C when played 

30 out over time. A second diagram 312 is obtained showing the evolution of a measure of the 
appreciation from the audience 302 along the content C when played out over time. For both 
diagrams 311 and 312, the appreciation is given in an arbitrary unit. Considering the diagram 
311, three peaks of the appreciation from the audience 301 are detected at locations Xll, 
X21 and X31 in the signal S. A set 321 of bookmarks BU, B12 and B13 are inserted into the 
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signal S at the locations Xll, X21 andXSl, respectively, as shown in Fig. 3. On the other 
hand, when considering the diagram 312, three peaks of the appreciation of the audience 302 
are detected along the content C. These peaks are located at locations X12, X22 and X32 in 
the content C or correspondingly in the signal S. A set 322 of bookmarks B21, B22 and B23 
S are inserted into the signal S at the locations X12, X22 and X32, respectively, as shown in 
Fig. 3. 

The use of set 321 or set 322 may be determined by the type of audience that is 
targeted by a display of the content C. Thus, the set 322 is used when the content C is played 
to an audience that has great similarities with the audience 302. For example, the audience 

10 302 may be composed of female auditors only. The audience 301 is mixed. A broadcaster, 
who desires to play the content C, which is a TV program, at a time when mostly female'are 
watching TV, uses the bookmarks B21, B22 and B23 to insert advertisements data AD21, 
AD22 and AD23 in the signal S. 

Furthermore the bookmarks B23 and B 13 are located at the same location X32 in the 

15 signal S. It is clear that the advertisement originated from the data AD23 inserted at this 
location X32 has a better chance of being viewed by a more auditors than an advertisement 
inserted at any another location in the content C. 

It is also within the scope of the invention to provide a signal S in which various sets 
of bookmarks have been inserted according to the appreciation of one or several audiences. 

20 For example, in a soccer match content C, a first set of bookmarks is inserted into the signal 
S for indicating the parts of the game when there is a scoring chance of one of the teams. A 
second different set of bookmarks is also inserted in the signal S for indicating the replays of 
the goals in the content C. A broadcaster supplies the signal S in which the two sets of 
bookmarks are embedded. A user, having previously recorded the signal S with the two sets 

25 of bookmarks provided by the broadcaster, has the possibility to choose to watch only the 
elements of the game indicated by the bookmarks of one of the two sets. Thus, the user can 
choose to use the first set of bookmarks and watches only the parts of the game when there 
was a scoring chance of one of the two teams. Or, the us^ can choose to use the second set of 
bookmarks and watches only the parts of the game where there is a replay of a goal. A 

30 broadcaster may choose to offer to sell the locations of the bookmarks of one set to a first 

advertisement provider and to offer to sell the locations of the bookmarks of the second set to 
another provider. 

It is to be noted that, with respect to the described method, modifications or 
improvements may be proposed without departing from the scope of the invention. For 
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instance, it is clear that a method of the invention may be implemented in several manners, 
such as by means of wired electronic circuits or, alternatively, by means of a set of 
instructions stored in a computer-readable medium, said instructions replacing at least a part 
of said circuits and being executable under the control of a computer or a digital processor in 
order to carry out the same functions as fulfilled in said replaced circuits. 
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CLAIMS: 



1. Method of using a bookmark (Bi) indicating an element in an information content (C), 
the element being determined from an appreciation from an audience (210) perceiving the 
information content. 

5 2. Method of Claim 1, wherein the information content (C) is displayed. 

3. Method of Claim 1, wherein the element is determined from known statistics of the 
audience's appreciation. 

10 4. Method of Claim 1, comprising: 

• supplying a first signal carrying the inforxnation content; 

• supplying a second signal comprising the bookmark. 

5. Method of Claim 1, wherein the using comprises inserting advertisement (ADi) into 
IS the information content (C) in the vicinity of the element. 

6. Method of Claim 1, wherein the using comprises inserting data into the infoimation 
content in the vicinity of the element. 

20 7. Method of Claim 6, wherein the information content comprises successive images and 
the data is a visual element inserted in an image. 

8, Method of Claim 6, wherein the information content is an audio and video program 
and the data is a video clip. 

25 

9. Method of Claim 6, wherein a broadcaster inserts the data before broadcasting the 
information content. 
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10. Method of Claim 1, wherein the bookmark is inserted in a signal (S) carrying the 
information content. 

11. Method of Claim 10, wherein a broadcaster inserts the bookmark (Bi) before 
5 broadcasting the signal (S). 

12. Method of Claim 10, wherein the bookmark is inserted in a prerecorded signal. 

13. Method of Claim 10, wherein the bookmark is inserted while recording the signal. 

10 

14. Apparatus for enabling to use at least one bookmark indicating an element in an * 
information content, the element being determined from an appreciation from an audience 
perceiving the information content. 

15 15. Apparatus of Claim 14, comprising: 

• recording means for recording a signal for allowing to play out the information content, 
and; 

• a marker for inserting the bookmark in the signal during the recording of the signal. 

20 16- Apparatus of Claim 14, comprising: 

• a player for playing out the infonxiation content from a signal, 

• a marker for inserting the bookmark in the signal during the play-out of the content. 

17. Storage medium for storing a digital signal into which a bookmark is inserted for 
25 indicating an element of an information content, the content being carried by the signal and 
the element being determined from an appreciation from an audience perceiving the 
information content. 



30 



18. Bookmarked signal carrying an information content, the signal comprising a 
bookmark for indication an element of the infonnation content, the element being determined 
from an appreciation from an audience perceiving the information content. 
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